Migrate build system from setuptools to maturin#105
Merged
dmartmillan merged 2 commits intoJun 11, 2026
Merged
Conversation
4 tasks
There was a problem hiding this comment.
Pull request overview
This PR migrates OpenVariant’s packaging/build pipeline from a pure-Python backend to maturin + PyO3, introducing a Rust-backed extension module (openvariant._core) and aligning tests/CI/docs with the new build and plugin conventions.
Changes:
- Switch
pyproject.tomlbuild backend tomaturinand add Rust crate scaffolding (Cargo.toml,src/) foropenvariant._core. - Update parsing/annotation delimiter handling (escape decoding + optional autodetection) and extend mapping builder to support absolute
fileMappingpaths. - Update plugin naming to PascalCase, add/adjust tests, and refresh multiple documentation examples/notebooks.
Reviewed changes
Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/test_task/test_plugin.py | Adds a regression test for PascalCase plugin/context class names in generated plugin templates. |
| tests/test_core_import.py | Adds a smoke test asserting openvariant._core is importable. |
| tests/test_annotation/test_builder.py | Adds coverage for absolute-path fileMapping in mapping annotations. |
| tests/test_annotation/test_annotation.py | Updates delimiter expectations for the annotation fixture. |
| tests/data/annotation/invalid_delimiter.yaml | Removes an invalid-delimiter fixture (previously used for delimiter validation behavior). |
| tests/data/annotation/annotation.yaml | Changes the test annotation delimiter to a tab escape (\\t). |
| src/lib.rs | Defines the PyO3 module entrypoint for _core and a sample hello function. |
| src/error.rs | Introduces a Rust error enum (OVError) for the new core crate. |
| pyproject.toml | Switches build backend to maturin, configures module name, and adds a docs entrypoint + pytest config. |
| openvariant/variant/variant.py | Changes base parsing to skip comments pre-split and to use delimiter autodetection / direct delimiter splitting. |
| openvariant/tasks/plugin.py | Adds PascalCase conversion for generated plugin class names. |
| openvariant/plugins/get_AF/get_AF.py | Renames plugin/context classes to PascalCase. |
| openvariant/plugins/get_AF/init.py | Updates exports to match renamed classes. |
| openvariant/plugins/alteration_type/alteration_type.py | Renames plugin/context classes to PascalCase. |
| openvariant/plugins/alteration_type/init.py | Updates exports to match renamed classes. |
| openvariant/plugins/init.py | Updates public exports (__all__) to match renamed plugin/context classes. |
| openvariant/annotation/config_annotation.py | Removes the delimiter enum/constants and leaves format enums only. |
| openvariant/annotation/builder.py | Supports absolute fileMapping paths and improves mapping-file resolution. |
| openvariant/annotation/annotation.py | Changes delimiter validation/handling to allow escape decoding and None delimiter for autodetection. |
| openvariant/_docs.py | Adds a small helper for building Sphinx docs via make html. |
| openvariant/init.py | Bumps package version to 2.0.0. |
| MANIFEST.in | Includes Rust sources/Cargo manifest in sdist and adjusts included/excluded paths. |
| Makefile | Adds convenience targets for cargo build/test/lint/format using uv’s Python. |
| docs/examples/variant/save.ipynb | Reformats notebook JSON/metadata and updates narrative cells. |
| docs/examples/variant/read.ipynb | Reformats notebook JSON/metadata and updates narrative cells. |
| docs/examples/tasks/group_by.ipynb | Reformats notebook JSON/metadata and updates narrative cells. |
| docs/examples/tasks/count.ipynb | Reformats notebook JSON/metadata and updates narrative cells. |
| docs/examples/tasks/cat.ipynb | Reformats notebook JSON/metadata and updates narrative cells. |
| docs/examples/plugin_system/get_length/get_length.py | Updates example plugin/context class names to PascalCase. |
| docs/examples/plugin_system/get_length/init.py | Updates example exports to match renamed classes. |
| docs/examples/plugin_system/add_date/add_date.py | Updates example plugin/context class names to PascalCase. |
| docs/examples/plugin_system/add_date/init.py | Updates example exports to match renamed classes. |
| docs/examples/plugin_examples.rst | Updates docs to match PascalCase plugin/context class names. |
| docs/examples/find_files/find_files_with_file_path.ipynb | Reformats notebook JSON/metadata and updates narrative cells. |
| docs/examples/find_files/find_files_with_directory_path.ipynb | Reformats notebook JSON/metadata and updates recorded Python version. |
| docs/examples/cli/main_cli.ipynb | Updates CLI examples and notebook structure/metadata. |
| docs/examples/cli/introduction_cli.ipynb | Updates CLI help examples and notebook structure/metadata. |
| CONTRIBUTING.md | Adds Rust toolchain/maturin developer setup instructions. |
| Cargo.toml | Adds Rust crate metadata for the PyO3 extension (cdylib) and dependencies. |
| Cargo.lock | Locks Rust dependencies for the new crate. |
| .gitignore | Ignores Cargo build artifacts (target/, etc.). |
| .github/workflows/openvariant_tester.yml | Installs Rust toolchain in CI and adjusts uv sync for linting. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
FedericaBrando
left a comment
FedericaBrando
left a comment
Member
There was a problem hiding this comment.
Great David!
I've run uv build (below the output), and then install the tar.gz in a clean conda environment. I tested the
>>> import openvariant._core as _core
...
... _core.hello()
...
Hello, OpenVariant!and worked, so, good to go!
╰╴❯ uv build
Building source distribution...
Running `maturin pep517 write-sdist --sdist-directory /home/fbrando/Desktop/bgtools/openvariant/dist`
🍹 Building a mixed python/rust project
🔗 Found pyo3 bindings
📡 Using build options features from pyproject.toml
From `cargo package --list --allow-dirty --manifest-path /home/fbrando/Desktop/bgtools/openvariant/Cargo.toml`:
warning: manifest has no description, license, license-file, documentation, homepage or repository
|
= note: see https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata for more info
📦 Built source distribution to /home/fbrando/Desktop/bgtools/openvariant/dist/open_variant-2.0.0.tar.gz
open_variant-2.0.0.tar.gz
Building wheel from source distribution...
Running `maturin pep517 build-wheel -i /home/fbrando/.cache/uv/builds-v0/.tmpuvOdC7/bin/python --compatibility off`
🍹 Building a mixed python/rust project
🐍 Found CPython 3.13 at /home/fbrando/.cache/uv/builds-v0/.tmpuvOdC7/bin/python
🔗 Found pyo3 bindings
📡 Using build options features from pyproject.toml
Compiling target-lexicon v0.13.5
Compiling proc-macro2 v1.0.106
Compiling unicode-ident v1.0.24
Compiling quote v1.0.45
Compiling libc v0.2.186
Compiling heck v0.5.0
Compiling thiserror v2.0.18
Compiling once_cell v1.21.4
Compiling pyo3-build-config v0.28.3
Compiling syn v2.0.117
Compiling pyo3-ffi v0.28.3
Compiling pyo3-macros-backend v0.28.3
Compiling pyo3 v0.28.3
Compiling thiserror-impl v2.0.18
Compiling pyo3-macros v0.28.3
Compiling openvariant-core v2.0.0 (/home/fbrando/.cache/uv/sdists-v9/.tmpiUVHkP/open_variant-2.0.0)
Finished `release` profile [optimized] target(s) in 10.15s
📦 Built wheel for CPython 3.13 to /home/fbrando/.cache/uv/sdists-v9/.tmpiUVHkP/open_variant-2.0.0/target/wheels/open_variant-2.0.0-cp313-cp313-linux_x86_64.whl
/home/fbrando/.cache/uv/sdists-v9/.tmpiUVHkP/open_variant-2.0.0/target/wheels/open_variant-2.0.0-cp313-cp313-linux_x86_64.whl
Successfully built dist/open_variant-2.0.0.tar.gz
Successfully built dist/open_variant-2.0.0-cp313-cp313-linux_x86_64.whlThe build works, there is a warning from cargo about the alck of metadata in the manifest that eventually we should address.
dmartmillan
deleted the
62-migrate-build-system-from-setuptools-to-maturin
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request corresponds to the Migrate build system from setuptools to maturin, be able to run Rust in Python scripts. Relates to #62 issue.
Most of the tasks for this PR were already solved on the previous one (#104)
On this PR, we:
MANIFEST.INto includeCargo.tomluv tool run maturin develop --releaseworks fine. (we should take into account for a future release)tests/test_core_import.pyand removedtest_rust.pymake releaseandmake testfor a future use.Closes #62